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(54) Communication system. 

@ A communication system comprising at least 
one mobile handheld telephone handset (1) 
adapted to communicate via a wireless tele- 
phony medium (3) with a telephone network 
handling system (3,4). 

The handset comprises input devices 
(10,8,11-13) to receive input from a user and 
produce signals dependent thereupon, an 
onboard processor (7) to adapt speech input to 
produce a voice transmission signal as part of a 
telephone conversation with a third party ; and 
an antenna (9) to transmit the voice transmis- 
sion signal via the wireless telephony medium 
(3). 

The telephone network handling system (3,4) 
comprises a receiver to receive the voice trans- 
mission signal, and means to forward the voice 
signal to a third party (29,30). 

The handset further comprises a first pro- 
cessor to carry out a first processing step on 
selected input signals and produce data depen- 
dent thereupon which preserves predetermined 
information necessary to carry out a remote 
second processing step, an onboard processor 
to adapt the data according to a conventional 
wireless telephony protocol to produce a trans- 
mission signal, and an antenna (9) to transmit 
the transmission signal via the wireless tele- 
phony medium to the telephone network hand- 
ling system. 

The system further comprises a remote pro- 
cessor (35,36,51) adapted to receive and adapt 
the transmission signal from the telephone net- 
work handling system to regenerate the data, 
and to carry out a second processing step on 
the data and produce an output (43) dependent 
thereupon. 
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' The present invention relates to a communication 
system. 

Systems such as Personal Digital Assistants, or 
PDAs are known which provide a variety of applica- 
tions including a diary, address book and computer 
applications such as wordprocessing and spread- 
sheets. PDAs are effectively "shrunken computers 0 - 
ie. they aim to provide the functionality of a desktop 
PC in a handheld device by using miniaturised com- 
ponents. Recent attempts have also been made to in- 
corporate communication features into a convention- 
al PDA, by adding mobile phone and/or fax capabili- 
ties. 

However, since a conventional PDA contains all 
systems and applications software/hardware on- 
board, the scope of the device is severely restricted 
due to size, processing capability and power supply 
limitations. 

In accordance with a first aspect of the present in- 
vention, we provide a communication system com- 
prising at least one mobile handheld telephone hand- 
set adapted to communicate via a wireless telephony 
medium with a telephone network handling system; 

wherein the handset comprises means to re- 
ceive input from a user and produce signals depend- 
ent thereupon, means to adapt speech input to pro- 
duce a voice transmission signal as part of a tele- 
phone conversation with a third party; and means to 
transmit the voice transmission signal via the wire- 
less telephony medium; and 

wherein the telephone network handling sys- 
tem comprises means to receive the voice transmis- 
sion signal, and means to forward the voice signal to 
a third party; 

characterised in that the handset further com- 
prises first processing means adapted to carry out a 
first processing step on selected input signals and 
produce data dependent thereupon which preserves 
predetermined information necessary to carry out a 
remote second processing step, means to adapt the 
data according to a conventional wireless telephony 
protocol to produce a transmission signal, and means 
to transmit the transmission signal via the wireless 
telephony medium to the telephone network handling 
system; and 

in that the system further comprises means to 
receive and process the transmission signal from the 
telephone network handling system to regenerate the 
data, and second processing means positioned re- 
mote from the handset and adapted to carry out a 
second processing step on the data, and to produce 
an output dependent thereupon. 

The system according to the present invention 
provides a means of delivering potentially unlimited 
processing power to a mobile handset, since the re- 
mote processor (which is generally fixed) is not limit- 
ed by size or power consumption requirements. 

The present invention typically includes a wire- 
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less telephone device for use in an associated wire- 
less telephony network environment (with the con- 
ventional "telephony" features of means to adapt 
speech input as part of a conventional telephone con- 

5 versation with a third party), along with the additional 
feature of further processing distributed between the 
handset and a remote processor which may be used 
in a variety of alternative applications. This enables 
effective human to computer interaction over a wire- 

10 less telephony medium by providing means in the 
handset to carry out a first processing step which, 
whilst processing, for example compressing, the input 
signals enough to enable acceptably error free trans- 
mission over the wireless telephony network, also 

15 preserves key information which is required to carry 
out a remote second processing step in a central 
processor, or remotely running application. Typically, 
the telephone network handling system comprises a 
wide area cellular network which utilises the GSM 

20 (General Standard Mobile) protocol, (with an associ- 
ated bandwidth for data transmission of 9.6 KBps). Al- 
ternatively, it may comprise the DCS 1800 or DECT 
(Digital European Cordless Telephony) environ- 
ments. 

25 The telephone network handling system is typi- 

cally a circuit switched system providing real-time 
communication between the handset and the third 
party (in the case of a telephone conversation), and 
between the first and second processing means (in 

30 the case of data transmission). This may be contrast- 
ed with a "packet switched" or "message switched" 
communication environment which can only be used 
for data transmission. 

Typically the means to process outgoing speech 

35 input for transmission and incoming voice signals for 
audio output comprises a modem, and may also fur- 
ther comprise a codec in the case of digital transmis- 
sion. 

The first and second processing steps may be 

40 consecutive steps in a speech recognition system. 
For instance, the first processing step in the handset 
may be the first step in a speech recognition process, 
and the second processing step may comprise the re- 
maining steps in the speech recognition process, pro- 

45 viding text/instructions as the output 

Typically the first processing step in the speech 
recognition process produces an output which retains 
key features of the input speech to allow the speech 
recognition process to be completed in a remote cen- 

so tral server, whilst also conforming with the bandwidth 
limitation of the wireless telephony medium (9.6KBbs 
in the case of GSM). 

Typically the first step in the speech recognition 
process comprises an initial feature analysis which 

55 converts the input speech signals according to a 
known algorithm (such as linear predictive coding) 
into parameter coefficients. The second processing 
step which is carried out in a remote central processor 
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typically comprises comparing the received parame- 
ter coefficients to a library of known sub-word units 
(such as phonemes, sub-phonemes or tri-phones) 
followed by further analysis against lexical, syntactic 
and semantic models to provide output 5 
text/instructions. 

The output from the central processor may be re- 
turned to the handset for display in the form of text, 
and may also be input as instructions to a further, re- 
motely processed application which accepts text as 10 
input 

The approach to voice processing recognises 
that significant resources, both in terms of processing 
power and data storage, are required by state-of-the- 
art recognition engines. Rather than delivering a com- 15 
promise solution through miniaturisation (as is the 
case with PDAs), limited in both vocabulary range 
and performance, the handset supports the initial sta- 
ges of voice recognition processing, i.e. the initial fea- 
ture analysis, on the handset, and passes the results, 20 
now sufficiently reduced in volume to fit the restricted 
data channel bandwidth offered by mobile/cordless 
networks, to a large, powerful, multi-user recognition 
engine located centrally. 

Alternatively, the first and second processing 25 
steps may be separate processes in a fax transmis- 
sion system, or first and second analysing steps in a 
handwriting recognition system, similar to the speech 
recognition system. 

In the case of a handwriting recognition system, 30 
text may be input to the handset via a touch sensitive 
screen and transmitted to the central processor using 
a standard facsimile protocol (or any other protocol 
which provides an appropriately high quality transfer 
method such that essential features in the written text 35 
are not lost). The central processor then carries out a 
handwriting recognition procedure on the fax data to 
interpret the handwriting and return it to the handset 
as recognised text. 

In a further alternative, the first and second proc- 40 
essing steps may be used in a "remote desktop" ap- 
plication -in which the handset acts as a remote in- 
put/output device for a computer. The first processing 
step typically comprises converting input from the 
user (for instance cursor control instructions) into ob- 45 
ject-level graphics code, tailored for a particular op- 
erating system (such as MS-Windows™ , Macin- 
tosh™ or any other object-oriented graphical inter- 
face) installed on a remote PC. The code is modulated 
according to the required wireless protocol and trans- so 
m it ted, via the telephone network handling system, to 
the remote PC. In this case the remote processor is 
contained in the PC, which decodes the object-level 
graphics code and inputs the instructions to a running 
application. 55 

Built-in intelligence in the handset identifies the 
input activity on an object basis, and transmits only a 
coded-representation of the object manipulation to 



the remotely executing application. Taking the exam- 
ple of drawing a line using a pointing device, the hand- 
set determines the type of object being input, in this 
case a line, and its properties (starting and ending co- 
ordinates, thickness, colour, etc.) which can then be 
transmitted in coded form to the application. The de- 
vice drivers associated with the remotely-executing 
application decode the input by reference to a stan- 
dard object library (which mirrors that held locally on 
the handset), and pass this into the application with 
no appreciable delay. The representation of the line 
shown on screen is generated locally by the handset, 
and only altered if the remote application dictates a 
change. To the user, the result is instantaneous re- 
sponse. 

In this case the application functionality available 
to the handset user is not limited by the capabilities 
of the on-board processor. In the "remote desktop" 
application, by communicating with and remotely con- 
trolling an application executing elsewhere, the hand- 
set delivers the power of that remote processor to the 
mobile user. Further, most users of the handset will 
already have a suite of applications, tailored to their 
information requirements, resident on their existing 
desktop PC. With the handset delivering access to all 
these resources from any location, there is no need 
for additional application software on the handset, 
with its cost and support implications, nor is a com- 
plex configuration exercise necessary in order to es- 
tablish connections to a range of information sources. 

As a further development of the "remote desktop" 
concept, a multi-user computer, capable of running 
application software which is commonly found on a 
personal computer, is located remotely within a tele- 
phone network. That is, the multi-user computer pro- 
vides application execution capability to a large pop- 
ulation of remote users as and when they need it. 
Handset users who are familiar with personal comput- 
er applications, but do not have an appropriately con- 
figured personal computer to access, can run these 
applications within a "virtual PC" environment on the 
multi-user node. This mechanism can deliver the ben- 
efits of personal computer applications to a handset 
user without requiring the capital outlay associated 
with a personal computer purchase. 

In accordance with a second aspect of the pres- 
ent invention, we provide a communication system 
comprising at least one mobile handheld telephone 
handset adapted to communicate via a wireless tel- 
ephony medium with a telephone network handling 
system; 

wherein the handset comprises means to re- 
ceive input from a user and produce signals depend- 
ent thereupon, means to adapt speech input to pro- 
duce a voice transmission signal as part of a tele- 
phone conversation with a third party; and means to 
transmit the voice transmission signal via the wire- 
less telephony medium; and 
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wherein the telephone network handling sys- 
tem comprises means to receive the voice transmis- 
sion signal, and means to forward the voice signal to 
a third party; 

characterised in that the handset is adapted to s 
transmit and receive input/output data via the wire- 
less telephony medium and the telephone network 
handling system whereby the handset acts as an in- 
put/output device for an application program execut- 
ing on a computer positioned remote from the hand- 10 
set and attached to the telephone network handling 
system. 

As previously described, the computer may be a 
desktop personal computer (PC) or a computer pro- 
viding a parallel multi-user environment. 15 

Again, the telephone network handling system is 
typically a circuit switched system. The limited band- 
width capabilities of the wireless telephony medium 
are typically overcome by converting input/output into 
an object-level graphics description protocol for 20 
transmission. 

An example of the present invention will now be 
described with reference to the accompanying Fig- 
ures in which:- 

Figure 1 illustrates a handset with associated 25 

network architecture; 

Figure 2 shows the exterior of the handset, with 

typical user input/output devices; 

Figure 3 illustrates the apparatus in a typical 

speech recognition process; 30 

Figure 4 is a flow diagram showing associated 

speech recognition steps; and, 

Figure 5 illustrates the apparatus in a remote 

desktop aspect of the present invention. 

The network architecture is shown in Figure 1, 35 
and comprises four key functional entities: handset 1 ; 
transport infrastructure 2 (signalling and bearer 
channels) comprising wireless 3 and fixed network 
transmission media 4; central processing, formatting 
and control platform 5; and service provision platform 40 
6. Item 6 is a generic entity representing assorted ser- 
vice provision platforms, which might include a store- 
and-forward fax server, interactive banking service or 
electronic mail access facility. 

A single processor 7 in the handset 1 is respon- 45 
sible for an overall control function, data processing 
and transaction functions associated with on- board 
applications, and DSP functions. The processor is 
typically a RISC based processor which offers high 
flexibility i.e. the chip power can be used for radio, 50 
computing applications, DSP or split between all 
these. The handset also supports a 16KHz micro- 
phone 8 with noise cancellation, and antenna 9. From 
hereon the processor 7 will be referred to as DSP 7, 
to differentiate it from other processors which will be 55 
described. 

The handset also comprises input and output de- 
vices as shown in Figure 2 for interaction with a user. 
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The handset comprises LCD display screen 10 for 
displaying fax, E-mail, menu options and other visual 
output. As can be seen in Figure 2, the area of the 
screen 10 occupies a significant proportion of the 
area of the handset 1 which is presented to the user. 
The handset also comprises input buttons 11,12 and 
a cursor control 13, along with the telephone micro- 
phone 8 and loudspeaker 14 which are positioned for 
optimal ear-to-mouth spacing. Typical dimensions of 
the handset are a length 15 of 170mm, a width 16 of 
70mm and a depth (not shown) of 23mm. This allows 
the handset to be easily held in the hand. The handset 
also comprises an antenna (not shown) which is inte- 
grated into the case and does not require extension 
for operation. 

The handset 1 may communicate with any con- 
ventional wireless communication environment 3, 
such as GSM 900 or DCS 1800. Alternatively, in a 
cordless system the mobile handset may communi- 
cate with a cordless (for instance DECT) base station. 
Apart from routing calls between the handset 1 and 
central platform 5, the transport infrastructure 2 may 
also set up telephone conversations between the 
handset 1 and a third party 29 or 30, either via other 
base stations in the wireless network 3, or via the 
PSTN 4 respectively. 

The central platform 5 typically comprises a pre- 
sentation manager 26. The presentation manager 26 
is responsible for the management and control of the 
interface between the central platform 5 and the 
handset 1 , and the acceptance of input from the de- 
vice. The presentation manager 26 also formats the 
output from any of the service provision platforms, 
and interprets events which occur within the hand- 
sets interface. The central platform 5 also carries out 
a variety of functions, including processing of Email 
and fax data which is forwarded/received from the 
handset 1. 

Multi-user database 32 is used to store data re- 
lating to each handset user. 

Item 28 represents a generic "service proxy*' ca- 
pability, which forms part of the central platform 5. 
This offers the "expected" interface to a service pro- 
vision platform 6, e.g. it emulates a VT100 terminal. 
Output is passed to the presentation manager 26 to 
be formatted for the handset, whilst events interpret- 
ed by the presentation manager 26 as requests to the 
service are appropriately translated and issued to the 
relevant platform 6. 

Voice processing host 35, is configured to partici- 
pate in a distributed speech recognition environment, 
and also incorporates voice synthesis capabilities to 
deliver audio output, such as prompts and instruc- 
tions, to the handset user. 

Handwriting recognition host 36 converts hand- 
written image data from the handset into text (descri- 
bed later). 

Atypical speech recognition application will now 
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be described with reference to Figures 3 and 4. 

Figure 3 shows the apparatus involved in a proc- 
ess of speech recognition. The first step in the speech 
recognition process is carried out in DSP 7 in the 
handset, and the remaining steps are carried out in 5 
voice processing host 35. The host 35 can deliver tex- 
tual output in an appropriate format in a manner sim- 
ilar to the output of a keyboard. The host 35 is respon- 
sive to speech input derived from the handset 1 by 
recognising speech units generated by the handset 10 
(eg. phonemes, sub-phonemes etc) and converting it 
into a textual representation. 

As will be appreciated, the mobile handset 1 is re- 
mote from the host 35 and this has, in the past, 
formed a substantial barrier to the provision of a fully- 15 
responsive high-quality speech recognition system 
by way of the currently available telephone network. 

As will be appreciated, in order to assist the host 
35 in attending to the required fully-responsive and 
high-quality recognition process, the speech informa- 20 
tion provided to the host 35 should carry high fidelity 
information similar to that achieved with CD or DAT 
audio systems. This requires a bandwidth in the order 
of 22 Khz and, although the host 35 is available to 
process the information provided in such a band- 25 
width, the communication link formed between the 
handset 1 and the speech recognition apparatus 35 
is not capable of providing such a bandwidth. 

The invention overcomes this problem by carry- 
ing out a unique analysis and compression process 30 
on the speech information captured by way of the mi- 
crophone 8. The analysed and compressed data only 
requires a bandwidth of an order that is commonly of- 
fered by present networks, for example 9.6 Kbps. The 
analysed and compressed data, once received by the 35 
host 35, is then used as a source for reconstructed 
speech data having an appropriately wide bandwidth, 
e.g. in the order of 22 Khz, which allows for the re- 
quired high-quality recognition procedure to be car- 
ried out at the host 35. 40 

Figure 4 is a flow diagram illustrating the various 
steps which are carried out by the apparatus ele- 
ments in Figure 3. 

The speech information that is to be subjected to 
the speech recognition process and subsequently to 45 
be employed as the control information for a remote 
system, is captured (step 60) by way of the micro- 
phone 8, having an appropriate transducer, and is fed 
to the DSP 7. The DSP 7 operates to capture and digi- 
tise the speech information (step 62) at high-fidelity so 
levels with a bandwidth in the order of 22 Khz. Sub- 
sequently, initial feature analysis (step 63) of the digi- 
tised speech signal is carried out in the DSP 7. Typical 
algorithms which may be employed at this stage are 
linear predictive coding or filter-bank analysis. The al- 55 
gorithm is chosen to preserve the full range of input 
frequencies (up to approximately 16KHz). This allows 
key features to be preserved after transmission, al- 
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lowing detailed (for instance, dictation quality) 
speech recognition at the host 35. The output of initial 
feature analysis comprises a stream of parameter 
coefficients 64. These coefficients may be coded 
further by the DSP if required to comply with the 
bandwidth of the transmission medium (for instance 
by temporal decomposition interpolation in a step not 
shown in Figure 4) and are then modulated at 65 in a 
conventional way to provide a radio transmission sig- 
nal according to the required protocol (for instance 
GSM, DECT or DCS 1800). The transmission signal 
is then passed to antenna 9 which transmits the sig- 
nal at 66. 

The transmitted signal 67 is then received and ra- 
dio demodulated in a conventional way by a base sta- 
tion 20 in the wireless telephone handling network 3 
and is eventually passed as digital data (via a number 
of units indicated generally by dotted lines 53) to the 
host 35. Depending on which protocol is used, the 
transmitted data is received at a Private Branch Ex- 
change (PBX) or a Mobile Switching Centre (MSC) 
before being delivered to the main communication 
network. The initial feature analysis parameter coef- 
ficients may have been regenerated in the previously 
described demodulation step. Alternatively, further 
decompression may be required by the host 35 to re- 
generate the initial feature analysis coefficients (if for 
example the coefficients had been coded by temporal 
decomposition interpolation). 

The coefficients are then used to look up a library 
41 of phonemes, sub-phonemes, diphones, tri-pho- 
nes or other sub-word units, depending on what the 
coefficients represent This is indicated at 69 in Fig- 
ure 4. The selected sub-word units are then analyzed 
by processor 42 at 70, which outputs text 71 which 
may be used in a number of ways. For instance, the 
textual output 43 may be retransmitted to the handset 
for display. In addition, the textual output may be used 
to provide input for a separately running application 
which recognises textual input 

Atypical speech recognition procedure has been 
outlined above, but in general any conventional 
speech recognition process may be used, with part of 
the process being carried out in the handset 1, and 
the rest in the central server 34. The division of tasks 
is determined to minimize loss of information and to 
optimize the use of the DSP 7. 

In one specific embodiment, the step of initial 
feature analysis 63 may involve generating data that 
is representative of speech elements, (e.g. pho- 
nemes, sub-phonemes etc.) of the information dign 
tised in the DSP 7. The following description shall de- 
scribe the use of phonemes, although the speech ele- 
ments may be sub-phonemes or tri-p hones. 

The data representative of the phonemes after 
encoding is arranged to be transmitted by the chosen 
protocol (e.g. DECT, GSM 900, DCS 1800 etc.). The 
DSP 7 contains a library of a large number of pho- 
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nemes which can be stored on approximately 2 M 
bytes of ROM. The encoded data signal derived from 
the DSP 7 for transmission via the chosen protocols 
is a highly accurate representation of the complete 
speech information captured by way of the micro- 5 
phone 8. The compression of the speech information 
thereby achieved by way of the DSP 7 is of particular 
relevance since transfer of the encoded data requires 
a far narrower bandwidth than was initially required to 
capture the speech data by way of the microphone 8. 10 
In particular, the bandwidth of the encoded data falls 
within the bandwidth that is available for transmission 
via the chosen protocol and via the Network PSTN in 
general. 

The encoded data representation of the pho- is 
nemes is then transmitted through the network with- 
out suffering any undue degradation and is directed 
from the gateway 34 to the voice processing host 35. 
This host 35 includes a library of data relating to all 
possible phonemes and mirrors the library of the DSP 20 
7. 

Thus, upon receipt of the encoded data transmit- 
ted via the network from the handset 1, the host 35 
is arranged to reconstruct the speech information by 
reference to the contents of its library. The recogni- 25 
tion process can be then carried out on a relatively 
high-fidelity speech information having the band- 
width of 22 Khz. 

The invention is particularly advantageous in that 
the speech recognition process eventually carried out 30 
in the host 35 is arranged to be speaker-specific since 
a smartcard level of user identification and security is 
inherent within the handset 1. As such, the central 
platform 5 and associated host 35 forms a powerful 
"recognition engine" which can be supplied with the 35 
necessary input information via the telephone net- 
work from a small and relatively inexpensive handset 
1 . Thus, the actual recognition procedure can be exe- 
cuted remote from such a handset and in an appro- 
priate environment that corresponds to specialized 40 
test conditions. Also, since the network sees the 
speech information merely as encoded data, full error 
correction and encryption can be applied where ap- 
propriate. Although this will cause response times to 
exceed the generally acceptable limit for con versa- 45 
tional speech, such variations do not present a signif- 
icant problem for speech recognition purposes in 
which mean delays of up to five seconds are generally 
considered acceptable. In any case, since phrases 
can be used as input instructions for information re- so 
trieval, greater delays, even of up to ten seconds, are 
regarded as reasonable. Primarily, a handset owner 
is provided with access to a speech recognition capa- 
bility that would not be otherwise available from a 
handset, or from a laptop PC, and would be particu- 55 
larly and prohibitively expensive in a dedicated desk- 
top machine. 

Also, by virtue of the architecture of the present 
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invention, the user of the handset 1 will be able to 
benefit directly from all improvements that are made 
to the central platform 5 and the host 35 as the related 
technology advances. 

As illustrated in Figure 1, the central platform 5 
can provide for access of the data appearing on the 
network for other applications 6 which might require 
vocal, textual or numerical commands such as the 
service operations of banks and retailers or other 
dedicated services provided on the network, for ex- 
ample the so-called Wings applications provided by 
Mercury Communications Limited. The level of secur- 
ity and user identification inherent in the use of the 
DSP 7 as an analysing and encoding device will prove 
particularly useful for such service operations. 

The provision of such high-quality speech recog- 
nition by way of a handset 10 and the PSTN provides 
the basis for a particularly advantageous multi-media 
data processing system. For example the combina- 
tion of voice response, speech recognition, speaker 
identification, text/image display, touch sensitive 
screen etc. provides for a system offering a wide va- 
riety of applications in a particular user-friendly and, 
responsive manner. 

A "remote desktop" application of the present in- 
vention will now be described with reference to Figure 
5. 

Existing protocols which support detached in- 
put/output require the transmission of a considerable 
amount of network traffic in order to complete, for ex- 
ample, the drawing of a line using a pointing device. 
The remotely executing application is supplied with 
input information reflecting each minute movement of 
the mouse, and responds with output instructions for 
each corresponding extension of the line to be dis- 
played on the screen. In practice any attempt to im- 
plement this conventional architecture on a mobile 
network would deliver considerable, unacceptable 
delay between the input action, and the resulting out- 
put. 

The "remote desktop" feature according to the in- 
vention allows the handset 1 to act as an input/output 
device for a remote desktop personal computer (PC), 
indicated generally at 50. 

The handset 1 sets up a call with the PC via a lo- 
cal base station 20 associated with a particular wire- 
less telephony network. The call is then forwarded to 
the PC via a telephone network handling system (in- 
dicated generally in dotted lines). 

As well as displaying screen information, the 
handset receives input from a user and thus acts as 
a "remote control" for the PC 50. This is achieved over 
a limited bandwidth communication medium by the 
use of an object-level graphics description protocol 
which is optimised for low bit-rate utilisation within the 
network which links the application (on the PC 50) 
and the handset 1. 

The PC comprises a processor 51 which runs an 
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application programme 52 supported by the Windows 
operating system 53. The Windows operating system 
53 receives input from a keyboard 54 and keyboard 
driver 55 which is typically converted into an object- 
level description protocol for input to the application 
programme 52. Similarly, output from the application 
programme 52 is converted into windows object-level 
function calls with referenced the appropriate win- 
dows libraries and output as function calls to screen 
driver 56 which drives the screen 57. The keyboard 
driver 55 and screen driver 56 are configured to input 
and output respectively information via a serial port 
58 which is connected to a network adaptor 71 con- 
nection to a telephone network. For instance the 
adaptor 71 may be an ISDN terminal adaptor. 

The keyboard driver 55 and screen driver 56 may 
be configured by the PC manufacturer to divert in- 
put/output function calls to/from the serial port 58 (as 
shown in Figure 5). Alternatively, a conventional PC 
may be adapted with a card inserted into the PC to en- 
able the interface with the remote handset In a fur- 
ther alternative, the PC 51 may be connected to a cor- 
porate server which has software which provides the 
interface between a number of PCs attached to it and 
the external telephone network. 

Turning now to the handset 1, the DSP 7 receives 
input from input means generally indicated at 70 
which may be the cursor control 13 or input from the 
touch sensitive screen 10. Built-in intelligence in the 
DSP 7 identifies the input activity on an object basis, 
and transmits only a coded representation (in appro- 
priate Windows™ format) of the object manipulation 
to the remotely executing application. Taking the line 
drawing example outlined above, the DSP 7 would 
determine the type of object being input with refer- 
ence to a Windows™ object library. In this case the 
DSP 7 determines that the object is a line, and also 
determines its properties (starting and ending coor- 
dinates, thickness, colour etc) which can then be 
transmitted in coded form to the application. The key- 
board and screen drivers 55,56 associated with the 
remotely executing application decode the input by 
reference to a standard object library (which mirrors 
that held locally on the handset) and pass this into the 
application with no appreciable delay. The represen- 
tation of the line shown on screen is generated locally 
by the DSP 7, and only altered if the remote applica- 
tion 52 dictates a change. To the user the result is in- 
stantaneous. 

When the remote application 52 dictates a 
change, screen driver 56 outputs the appropriate 
function call which is transmitted to the handset 1 via 
adaptor 71 , and telephone network Base Station Con- 
troller 20. The DSP 7 receives the signal and process- 
es it with reference to the stored standard object libra- 
ry. Appropriate output is then displayed on the screen 
10. 

Instead of accessing a conventional PC as shown 
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in Figure 5, the handset may access a "multi-user 0 
which provides application execution capability to a 
large population of users in parallel, as and when they 
need it. 

5 The use of the handset as an Input/Output and 

control device for a store-and-forward fax service will 
now be described. 

A store-and-forward fax service acts as a mail- 
box for facsimile messages. A centrally connected 

10 node offers a line-terminating interface to which in- 
coming fax messages are routed e.g. via call- 
forwarding or as part of an intelligent peripheral ar- 
rangement (such as Mercury's Wings platform). 
These incoming faxes pass through the defined 

15 fax interface and are stored on disk in digital form. Re- 
ceipt of the fax may trigger a notification process, 
alerting the user to the presence of new messages, 
via a pager message, for example. 

Access to stored messages is traditionally ach- 

20 ieved via a telephone handset DTMF interface, 
through which a suitable destination fax number is in- 
put by the user. On specification of the fax number, 
the mailbox is "unloaded", through conversion of the 
stored digital data across the defined fax interface, 

25 and transmission in analogue form. 

The handset's large, touch-sensitive screen al- 
lows it to act as a terminal for the input of handwritten 
or sketched faxes, while its scrolling display capabil- 
ity allows externally originating faxes to be read. In 

30 addition, the ability to participate in a distributed 
speech recognition environment will support spoken 
text or command input. 

The combination of the handset and a store-and- 
forward fax service offers an effective solution to mo- 

35 bile faxing requirements. The gateway can receive 
large faxes on the user's behalf and alert the user via 
the handset's graphical user interface. This can indi- 
cate the sender's details (extracted from the fax 
header) along with the message size. A user can then 

40 choose to view the message on the handset's screen, 
scrolling down each page, or issue a command for the 
fax to be forwarded to a nearby fax machine if the 
message is large or a hard copy is required. 

Using the touch sensitive screen, the user can 

45 annotate an incoming fax, perhaps signing it as au- 
thorisation, for example, and return it to the sender or 
forward to another party. Re-trying busy numbers 
and forwarding to multiple destinations are features 
supported within the store-and-forward environment, 

so thus reducing the amount of time required to process 
and reply to messages. 

In most instances, the transfer of facsimile infor- 
mation between the handset and the service host will 
use a standard fax protocol. Group III fax is included 

55 within the GSM phase 2 standard, and is also support- 
ed within many other cellular and cordless environ- 
ments. However, there is an opportunity for small 
messages, such as a screen-sized "fax note", to be 

7 
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transmitted via a proprietary protocol over a standard 
transport mechanism. As an example, Apple and Mo- 
torola have developed a compression standard which 
can represent "raw ink" (handwritten text and sketch- 
es) from an Apple Newton Message Pad screen in 5 
around 500 bytes, originally as part of their work on 
two-way paging around Asia. 

Given that the SMS facility for out of band mes- 
saging within GSM and DCS 1800 (and the COMS fa- 
cility within DECT) offers a throughput around 160 10 
bytes per message, a fax note could be sent over 4 
such messages. Using this method to move the fax 
representation from the handset to the service host 
would eliminate the expense of a switched bearer 
channel, and would also allow a call in progress to 15 
continue simultaneously. 

A method of implementing a distributed handwrit- 
ing recognition environment will now be described. 

Conventional Personal Digital Assistants have 
attempted to allow text input via handwriting recogni- 20 
tion. This process involves the use of local resources 
to interpret handwritten text entered on a touch- 
sensitive screen. Unfortunately, this has not per- 
formed particularly effectively. 

The availability of more powerful remote proc- 25 
essing resources, along with more extensive refer- 
ence data, significantly improves the performance of 
the handwriting recognition process. Therefore a dis- 
tributed system according to the invention, in which 
the handset gathers and presents data for processing 30 
by a centrally located host provides significant advan- 
tages. 

In a similar fashion to the voice processing sys- 
tem described previously, the input data needs to be 
transmitted to the central resources without losing 35 
any features which contribute to the accuracy of proc- 
essing. Handwriting can be recognised on the basis 
of dynamic features, and static graphic features. Dy- 
namic features may be analysed at the handset by the 
DSP 7, and transmitted to a remote handwriting rec- 40 
ognition processor 36 along with graphic features. 

Preferably the graphic features are transmitted 
via a protocol which retains predetermined features of 
the input handwriting signals to allow the handwriting 
recognition process to be completed in the remote 45 
processor. 

Standard facsimile protocols offer an appro- 
priately high quality transfer method for graphic fea- 
tures, such that the handset (or any other piece of 
equipment which conforms to standard facsimile 50 
standards) can deliver handwriting for recognition via 
fax, and display the recognised output. 



Claims 



55 



to communicate via a wireless telephony medium 
(3) with a telephone network handling system 
(3.4); 

wherein the handset comprises means 
(10,8,11-13) to receive input from a user and pro- 
duce signals dependent thereupon, means (7) to 
adapt speech input to produce a voice transmis- 
sion signal as part of a telephone conversation 
with a third party; and means (9) to transmit the 
voice transmission signal via the wireless teleph- 
ony medium (3); and 

wherein the telephone network handling 
system (3,4) comprises means to receive the 
voice transmission signal, and means to forward 
the voice signal to a third party (29,30); 

characterised in that the handset further 
comprises first processing means (7) adapted to 
carry out a first processing step on selected input 
signals and produce data dependent thereupon 
which preserves predetermined information nec- 
essary to carry out a remote second processing 
step, means to adapt the data according to a con- 
ventional wireless telephony protocol to produce 
a transmission signal, and means (9) to transmit 
the transmission signal via the wireless teleph- 
ony medium to the telephone network handling 
system; and 

in that the system further comprises 
means to receive and process the transmission 
signal from the telephone network handling sys- 
tem to regenerate the data, and second process- 
ing means (35,36,51) positioned remote from the 
handset and adapted to carry out a second proc- 
essing step on the data, and to produce an output 
(43) dependent thereupon. 

2. A system according to claim 1 wherein the first 
and second processing means are adapted to 
carry out first (63) and second (69,70) steps in a 
speech recognition process. 

3. A system according to claim 2 wherein the select- 
ed input signals comprise speech signals, and 
wherein the first processing means provides an 
output (64) which retains predetermined features 
of the input speech signals to enable the second 
processing means (35) to carry out the remaining 
steps of the speech recognition process. 

4. A system according to claim 3, wherein the sec- 
ond processing means (35) provides an output 
(43) comprising text signals representative of the 
textual content of the input speech signals, and 
returns the text signals to the handset for output 
as text. 



1. A communication system comprising at least one 
mobile handheld telephone handset (1) adapted 



5. A system according to claim 3 or 4 wherein the 
first processing step comprises initial feature 
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analysis. 

6. A system according to any of the preceding 
claims, wherein the first and second processing 
means are adapted to carry out first and second 
steps in a handwriting recognition process. 

7. A system according to claim 6 wherein the select- 
ed input signals comprise handwriting signals in- 
put via a touch sensitive screen (1 0) on the hand- 
set, and wherein the first processing means pro- 
vides an output which retains predetermined 
graphic features of the input handwriting signals 
to enable the second processing means (36) to 
carry out the remaining steps of the handwriting 
recognition process. 



10 



15 



sion signal as part of a telephone conversation 
with a third party; and means (9) to transmit the 
voice transmission signal via the wireless teleph- 
ony medium (3); and 

wherein the telephone network handling 
system (3,4) comprises means to receive the 
voice transmission signal, and means to forward 
the voice signal to a third party (29,30); 

characterised in that the handset is adapt- 
ed to transmit and receive input/output data via 
the wireless telephony medium and the tele- 
phone network handling system whereby the 
handset acts as an input/output device for an ap- 
plication program executing on a computer posi- 
tioned remote from the handset and attached to 
the telephone network handling system. 



8. A system according to claim 7 wherein the trans- 
mission signal is modulated according to a con- 
ventional facsimile protocol. 



20 



16. A system according to claim 15, wherein output 
from the remotely executing application is trans- 
mitted to be displayed on the handset. 



9. A system according to claim 7 or 8 wherein the sec- 
ond processing means provides an output compris- 
ing text signals representative of the textual content 
of the input handwriting signals, and returns the text 
signals to the handset for output as text 

10. A system according to claim 4 or 9 wherein the 
text is input as instructions to a further remotely 
processed application. 

11. A system according to any of the preceding 
claims, wherein the handset acts as a remote in- 
put device for a computer (50). 

12. A system according to claim 11 wherein the first 
processing step comprises converting the select- 
ed input signals into object-level graphics code 
suitable for the computer (50). 

13. A system according to claim 12 wherein the com- 
puter comprises a multi-user computer. 

14. A system according to claim 12 or 13 wherein the 
second processing means (3) is adapted to con- 
vert the object-level graphics code into input for 
an application programme (52) running on the 
computer. 

15. A communication system comprising at least one 
mobile handheld telephone handset (1) adapted 
to communicate via a wireless telephony medium 
(3) with a telephone network handling system 
(3,4); 

wherein the handset comprises means 
(10,8,11-13) to receive input from a user and pro- 
duce signals dependent thereupon, means (7) to 
adapt speech input to produce a voice transmis- 



17. A system according to claim 15 or 16 wherein se- 
lected input signals from the user are transmitted 
as an output signal to the computer for input to the 

25 remotely executing application. 

18. Asystem according to claim 16 or 17 wherein the 
input and/or output signals are in the form of an 
object- level graphics description protocol. 



30 
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19. A system according to any of claims 15 to 18 
wherein the computer executes a plurality of ap- 
plication programs each application program be- 
ing accessed by a different remote handset. 

20. A system according to any of the preceding 
claims, wherein the telephone network handling 
system comprises a circuit switched system. 



40 21. A method of telephonic speech recognition in a 
system according to any of claims 1 to 14, where- 
in said first processing step comprises analysing 
speech signals obtained from speech input, ana- 
lysing said speech signals so as to identify speech 

45 elements of said speech input and producing data 

representative of said speech elements of said 
speech input and wherein said second processing 
step comprises conducting a speech recognition 
procedure on the basis of the speech elements rep- 

so resented by said transmitted data. 

22. A method according to claim 21, wherein said 
speech elements comprise phonemes. 

55 23. A method according to claim 21, wherein said 
analysis of said speech signals is conducted at a 
sub-phoneme level. 



BNSDOCID: <EP_0671721A2_I_> 




BNSDOCID: <EP 0671721 A2_l_> 




/ \/V / V 



EP 0 671 721 A2 



Fig.3. 

20 } 



Fig.4. 




MICRO- 
PHONE 
8 



DSP < 
7 



COLLECT 
SPEECH 



^ — 61 



DIGITISE 



ANTENNA* 
9 



BASE- < 
STATION I 
20 



PERFORM INITIAL 
FEATURE 
ANALYSIS 
(eg LPC) 



-64 



MODULATE I FA 

PARAMETER 
COEFFICIENTS 



60 



62 



.63 



71 



70 



65 



.43 



OUTPUT 
WORDS 



ANALYSE PHONEMES 
/ SUB-PHONEMES 



GSM/ 
i DECT/DCS 



TRANSMIT 



■67 



66 



PROCESSOR 
42 



LOOK-UP PHONEMES /SUB- 
PHONEMES IN LIBRARY 41 
USING I FA PARAMETER 
COEFFICIENTS 



RECEIVE AND 
DEMODULATE 



IFA PARAMETER 
COEFFICIENTS 



69 



68 



-LIBRARY 
41 



11 



BNSDOCIO: <EP 0671721 A2_l_> 



EP 0 671 721 A2 



Fig.5. 

70 




12 



BNSDOCID: <EP 0671721 A2_l_> 



(19) 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 



(12) 



(11) EP 0 671 721 A3 

EUROPEAN PATENT APPLICATION 



(88) Date of publication A3: 

21.02.1996 Bulletin 1996/08 

(43) Date of publication A2: 

13.09.1995 Bulletin 1995/37 

(21) Application number: 95300954.5 

(22) Date of filing: 15.02.1995 



(51) mtci.6; G10L 3/00, G10L 9/00, 
H04M 1/72 



(84) 


Designated Contracting States: 


(72) 


Inventors: 




AT BE CH DE DK ES FR GB GR IE IT LI LU MC 


• 


Moore, lain Charles 




NL PT SE 


• 


Crewe, Cheshire CW1 4TG (GB) 
Ellis, Christopher William Henderson 


(30) 


Priority: 10.03.1994 GB 9404620 
21.12.1994 GB 9425836 


(74) 


Leamington, SpaCV32 5NW (GB) 
Representative: Skone James, Robert Edmund 


(71) 


Applicant: CABLE & WIRELESS PLC 
London WC1 X 8RX (GB) 




London EC2M 7LH (GB) 



(54) Communication system 



(57) A communication system comprising at least one mobile handheld telephone handset (1 ) adapted to commu- 
nicate via a wireless telephony medium (3) with a telephone network handling system (3,4). 

The handset comprises input devices (10,8,11-13) to receive input from a user and produce signals dependent 
thereupon, an onboard processor (7) to adapt speech input to produce a voice transmission signal as part of a telephone 
conversation with a third party; and an antenna (9) to transmit the voice transmission signal via the wireless telephony 
medium (3). 

The telephone network handling system (3,4) comprises a receiver to receive the voice transmission signal, and 
means to forward the voice signal to a third party (29,30). 

The handset further comprises a first processor to carry out a first processing step on selected input signals and 
produce data dependent thereupon which preserves predetermined information necessary to carry out a remote second 
processing step, an onboard processor to adapt the data according to a conventional wireless telephony protocol to 
produce a transmission signal and an antenna (9) to transmit the transmission signal via the wireless telephony medium 
to the telephone network handling system. 

The system further comprises a remote processor (35,36,51 ) adapted to receive and adapt the transmission signal 
from the telephone network handling system to regenerate the data, and to carry out a second processing step on the 
data and produce an output (43) dependent thereupon. 
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LACK OF UNITY OF INVENTION 




or.rch Oivts.on considers that the present Eurocean patent application does not conip.y with the requirement of unity of 
ion nro relates to several inventions or Qrcups of inventions. 



1. Claims i-5,20 (as far as depending on 1-5), 
21-23 (as far as depending on 1-5): 

Splitting of speech recognition process between 
handset and exchange 

2 . Claims 6-10,20 [as far as depending on 6-10), 
21-23 (as far as depending on 6-10): 

Handwriting recognition split between handset and 
exchange 

3. Claims 11-19,20 (as far as depending on 11-19*, 21-23 
(as far as depending on 11-14): 

: Jrie of hands en as a remote control 
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